United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
I nilid Stall-, l'atint and Trademark Office 

Address: COMMISSIONER FOR PATENTS 



APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO. 



Robert Mark Stefan Porter 282M61 S8X1>< I 



22850 7590 08/01/2008 FXAMTNFR 

OBLON, SPIVAK, MCCLELLAND MAIER & NEUSTADT, P.C. I 

1 940 DUKE S TREET redding, thomas m 

ALEXANDRIA, VA 223 14 I 



PAPER NUMBER 



NOTIFICATION DATE | DELIVERY MODE 
08/01/2008 ELECTRONIC 



Please find below and/or attached an Office communication concerning this application or proceeding. 

The time period for reply, if any, is set in the attached communication. 

Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the 
following e-mail address(es): 

palcnldockcKo'oblon.coin 
oblonpat@oblon.com 
j gardner @ oblon. com 



PTOL-90A (Rev. 04/07) 



l/ffflrC? nVrliUli Otfff Iff ids y 


Application No. 

10/536,697 


Applicant(s) 

PORTER ET AL. 


Examiner 

THOMAS M. REDDING 


Art Unit 

2624 





- The MAILING DATE of this communication appears on the cover sheet with the correspondence address — 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )KI Responsive to communication(s) filed on 10 April 2008 . 
2a )^ This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1-4,6-11,14 and 16 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) |EI Claim(s) 1-4,6-11.14 and 16 is/are rejected. 

7) 0 Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) Q The specification is objected to by the Examiner. 

10) ^ The drawing(s) filed on 27 May 2005 is/are: a)^ accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) ^ Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)E| All b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

20 Certified copies of the priority documents have been received in Application No. . 

3.^ Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attach ment(s) 

1) ^| Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-41 3) 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) Paper No(s)/Mail Date. . 

3) □ Information Disclosure Statement(s) (PTO/SB/08) 5 ) □ Notice of Informal Patent Application 

Paper No(s)/Mail Date . 6) □ Other: . 



PTOL-T26 d (Rev e 08-06r 



Office Action Summary 



Part of Paper No./Mail Date 20080728 



Application/Control Number: 10/536,697 
Art Unit: 2624 



Page 2 



DETAILED ACTION 

Response to Amendment 

1 . Applicant's response received on 4/10/2008 is fully considered herein. Claims 1- 
4, 6-11, 14 and 16 are currently pending, claims 5, 12-13 and 15 have been cancelled 
by the applicant. 

Specification 

2. In response to applicant's amendment of the abstract removing the title and 
extraneous text the objection to the abstract is withdrawn. 

Claim Rejections - 35 USC § 101 

3. In response to applicant's cancellation of claims 1 2-1 3 and amendment of 
claim14 to a statutory form, the 101 rejection is withdrawn. 

Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 
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5. Claims 1,2, 4, 6, 8-11, 14 and 16 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Jones et al. (US 7,099,510) in combination with McGrath et al. (US 
7,289,717). 

Regarding claims 1,11, 14 and 16, Jones teaches [a] face detection apparatus 
("the object detector of the present invention detects objects (e.g., faces) at any scale 
and location", Jones, column 2, line 36) generating an output indicating a likelihood of a 
region of an image containing a face, the apparatus comprising: 

means for (structure disclosed by the applicant is a general purpose computer, 
page 3, line 8, Jones likewise discloses "the present invention provides computer 
apparatus and computer-implemented methods for detecting instances of objects in an 
image", Jones, column 3, line 32), comparing the region with face data that indicates a 
presence of the face and for detecting respective likelihood values indicating the 
likelihood of the region containing the face in a group of respective different face sizes 
("Using a cascaded approach, the present invention quickly determines if a face could 
potentially appear at a given scale and location", Jones, column 2, line 53 and "The 
summation function includes a global threshold that determines whether or not a sum of 
the summation function indicates a detection of one of the instances of the certain 
object in the given subwindow", Jones, column 4, line 32, Jones calculates values for 
given classification scenarios); and 

means for (structure disclosed by the applicant is a general purpose computer, 
page 3, line 8, Jones likewise discloses "the present invention provides computer 
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apparatus and computer-implemented methods for detecting instances of objects in an 
image", Jones, column 3, line 32) modifying at least one of the group and the likelihood 
values based on a face size or a range of face sizes appropriate to at least one 
classification of the image ("This approach of the invention allows the object detector to 
discard quickly subwindows that do not show enough features of the object and to 
continue to process through the cascade only those subwindows that have enough 
features that indicate the likelihood of an instance of the object in the subwindow", 
Jones, Column 2, line 60, The output from the previous stage affects the evaluation of 
subsequent stages). 

Jones does not teach the means for modifying being responsive to metadata 
associated with the image defining the at least one classification of the image, the 
image being a part of a video sequence and the at least one classification includes a 
video program type. 

McGrath working in a related problem solving area of video acquisition does 
teach metadata associated with the image ("The term metadata as used herein refers to 
and includes any form of information or data which serves to describe either the content 
of audio/video material or parameters present or used to generate the audio/video 
material or any other information associated with the audio/video material", McGrath, 
column 1, line 48) defining the at least one classification of the image ("Metadata may 
be, for example, "semantic metadata" which provides contextual/descriptive information 
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about the actual content of the audio/video material. Examples of semantic metadata 
are the start of periods of dialogue, changes in a scene, introduction of new faces or 
face positions within a scene", McGrath, column 1, line 53, The semantic information 
described by McGrath does classify the nature of the material covered in the sequence), 
the image being a part of a video sequence ("FIG. 7 provides a schematic 
representation of a post production process in which the audio/video material is edited 
to produce an audio/video program", McGrath, column 1 1 , line 62) and the at least one 
classification includes a video program type (McGrath, table in column 7 describes type 
of content of the video for different shots). 

It would have been obvious at the time the invention was made for one of 
ordinary skill in the art to use the method of McGrath to record and associate metadata 
with video sequences together with the face detection system of Jones to select only 
relevant video sequences of interest ("As such, the editing terminal 184 can be used to 
identify and select from the metadata 210 the items of audio/video material which are 
required in order to produce a program", McGrath, column 14, line 14. Jones teaches 
the concept of using sub-windows to quickly determine regions in an image that could 
contain a face; the teaching of McGrath extends this concept to the next level to only 
consider video sequences that likely contain faces based on hints provided by the 
associated metadata). 
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Further regarding claim 14 and 16, Jones teaches a computer readable medium 
including computer executable instructions, wherein the instructions when executed by 
a processor cause the processor to perform the method described above ("In one 
embodiment, a computer program product 180, including a computer readable or usable 
medium (e.g., one or more CDROM's, diskettes, tapes, etc.), provides software 
instructions for the image integrator 26 and/or object detector 28. The computer 
program product 180 may be installed by any suitable software installation procedure, 
as is well known in the art", Jones, column 6, line 28 and with respect to claim 16, the 
programming described by Jones provides the configuration of the comparator and 
controller). 

Regarding claim 2, the combination of Jones and McGrath teaches wherein the 
means for comparing weighs the likelihood values to enhance the likelihood of the face 
in a face size appropriate to the at least one classification of the image ("the present 
invention quickly determines if a face could potentially appear at a given scale 
Jones, column 2, line 53 and "The object detector scales the homogenous classification 
functions respectively for each different size of the working window", Jones, column 3, 
line 56, if the face cannot appear at a given scale, it's likelihood is effectively zero). 

Regarding claim 4, the combination of Jones and McGrath teaches means for 
detecting whether a greatest one of the likelihood values exceeds a threshold likelihood 
value ("The summation function includes a global threshold that determines whether or 
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not a sum of the summation function indicates a detection of one of the instances of the 
certain object in the given subwindow", Jones, column 4, line 32). 

Regarding claim 6, the combination of Jones and McGrath teaches wherein the 
comparing means for comparing derives attributes from respective blocks of the region 
("Each of the homogenous classification functions in sequence in the cascade 
respectively has increasing accuracy in identifying the certain objects. A homogenous 
classification function consists of a number of features", Jones, column 3, line 41 ), 
compares the derived attributes with attributes that indicate the presence of the face, 
and derives a probability of the presence of the face by a similarity between the derived 
attributes and the attributes that indicate the presence of the face ("The summation 
function includes a global threshold that determines whether or not a sum of the 
summation function indicates a detection of one of the instances of the certain object in 
the given subwindow", Jones, column 4, line 32). 

Regarding claim 8, the combination of Jones and McGrath teaches [a] video 
conferencing apparatus: comprising the apparatus according to claim 1 ("Knowledge of 
the location and scale of a face can be used in teleconferencing applications as well", 
Jones, column 3, line 10). 
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Regarding claim 9, the combination of Jones and McGrath teaches [a] 
surveillance apparatus comprising the apparatus according to claim 1 ("A face detector 
based on the present invention can also play a central role in security camera 
applications. Such a face detector may be used to summarize many hours of airport 
surveillance tape into a single web page that shows a picture of each person that 
passed through a security checkpoint", column 3, line 15)). 

Regarding claim 10, the combination of Jones and McGrath teaches [a] display 
apparatus comprising: a display screen; a video camera; and the apparatus according 
to claim 1, wherein 

the video camera and the apparatus are arranged with respect to the display screen to 
detect faces looking at the display screen ("The present invention can be used in real- 
time applications in which the appearance of an object can be used to drive a user 
interface. For example, an object detector for faces (i.e., face detector) that is designed 
in accordance with the present invention functions in a kiosk like those used in bank 
ATM's (automatic teller machines) or airport ticketing machines to identify faces in a 
real-time application", Jones, column 3, line 4). 

6. Claim 3 is rejected under 35 U.S.C. 103(a) as being unpatentable over Jones et 
al. (US 7,099,510) and McGrath et al. (US 7,289,717) in combination with Daly et al. 
(US 6,173,069). 
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Regarding claim 3 the combination of Jones and McGrath teaches an [apparatus 
according to claim 1. 

The combination of Jones and McGrath does not teach wherein the means for 
modifying selects a subset of the group of for detecting the likelihood values, and the 
subset is dependent upon the face size or the range of face sizes appropriate to the at 
least one classification of the image. 

Daley, working in the same field of endeavor of face detection does teach 
wherein the means for modifying selects a subset of the group of for detecting the 
likelihood values, and the subset is dependent upon the face size or the range of face 
sizes appropriate to the at least one classification of the image ("In most video 
telephone applications the head is usually centrally located in the upper third of the 
image. Moreover, the size of the face is usually within a range of sizes and thus 
candidate circles that are exceedingly small or excessively large are not suitable", Daly, 
column 7, line 45). 

It would have been obvious to one of ordinary skill in the art to use the teaching 
of Daly to select a sub-set of possible face positions and sizes with the face detection 
system of Jones in order to use the "implicit characteristics [of video teleconferencing 
devices] that may be exploited to further determine the appropriateness of candidate 
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circles" (Daly, column 7, line 43). Imposing these limits also reduce the amount of 
computation required to analyze a scene. 

7. Claim 7 is rejected under 35 U.S.C. 103(a) as being unpatentable over Jones et 
al. (US 7,099,510) and McGrath et al. (US 7,289,717) in combination with Moghaddam 
et al. (IEEE, TPAMI Vol 19, 1997). 

The combination of Jones and McGrath teaches an [apparatus according to 
claim 6. 

The combination of Jones and McGrath does not teach wherein the attributes 
that indicate the presence of the face comprise projections of image areas onto one or 
more image eigenblocks. 

Moghaddam, working in a similar problem solving area of face recognition does 
teach wherein the attributes that indicate the presence of the face comprise projections 
of image areas onto one or more image eigenblocks ("we must incorporate the 
underlying probability distribution of the object. Subspace methods and eigenspace 
decompositions are particularly well-suited to such a task", Moghaddam, page, 696, 
Section 1 .1 , first paragraph). 
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It would have been obvious at the time the invention was made for one of 
ordinary skill in the art to use the eigen-space method as taught by Moghaddam in the 
face detection system of Jones to implement a face recognizer that can provide a 
compact and parametric description of the object's appearance and also automatically 
identify the degrees-of-freedom of the underlying statistical variability (Moghaddam, 
page 697, first paragraph). It would be an additional classification method to add to 
Jones's cascade of classifiers. 
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Response to Arguments 



Summary of Applicant's Remarks: With respect to claims 1 , 11, 14, and 1 6 Jones 
does not teach or suggest modifying likelihood values based either on face size or 
range of face size appropriate to classification. 

Examiner's Response: Jones determines if a region could contain a face ("Using a 
cascaded approach, the present invention quickly determines if a face could potentially 
appear at a given scale and location", Jones, column 2, line 53). If no face is determined 
to be there (in a specified size), the likelihood is low or zero. 

Summary of Applicant's Remarks: With respect to amended claims 1, 11, 14, and 16, 
the Jones reference does not teach or suggest a means for modifying the group of 
respective face sizes or corresponding likelihood score of the faces in the group 
responsive to metadata associated with an image defining at least one classification of 
the image. 

Examiner's Response: See updated rejection above. McGrath teaches the use of 
metadata describing video content. 



Summary of Applicant's Remarks: With respect to amended claims 1, 11, 14, and 16, 
the Jones reference does not teach or suggest a classification includes a program type. 
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Examiner's Response: See updated rejection above. McGrath teaches the use of 
metadata describing video content. 

Summary of Applicant's Remarks: With respect to amended claims 1, 11, 14, and 16, 
the Daley reference is silent with regards to metadata associated with an image defining 
at least one classification of the image. 

Examiner's Response: See updated rejection above. McGrath teaches the use of 
metadata describing video content. 



Summary of Applicant's Remarks: One of ordinary skill in the art wishing to improve 
the speed of the system disclosed by Jones would be unlikely to consider Daly which 
describes a relatively crude system that would have problems with false positives and 
false negatives, and makes overly limiting assumptions regarding the form the 
silhouette of a face would take. 

Examiner's Response: The Daly and Jones references both deal with a common 
problem of face detection and would be of interest to one of ordinary skill in the art 
working on related problems. In particular, the teaching to use relatively simple tests to 
quickly eliminate low probability candidates would be of interest for the potential 
increase in system performance. 
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Conclusion 

8. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to THOMAS M. REDDING whose telephone number is 
(571)270-1579. The examiner can normally be reached on Mon - Fri 7:30 am - 5:00 pm 
EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Vikkram Bali can be reached on (571) 272-7415. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



IT. M. RJ 

Examiner, Art Unit 2624 
A/ikkram Bali/ 

Supervisory Patent Examiner, Art Unit 2624 



